YORC: Yoruba Reading Comprehension dataset

In this paper, we create YORC: a new multi-choice Yoruba Reading Comprehension dataset that is based on Yoruba high-school reading comprehension examination. We provide baseline results by performing cross-lingual transfer using existing English RACE dataset based on a pre-trained encoder-only model. Additionally, we provide results by prompting large language models (LLMs) like GPT-4

How good are Large Language Models on African Languages?

Recent advancements in natural language processing have led to the proliferation of large language models (LLMs). These models have been shown to yield good performance, using in-context learning, even on unseen tasks and languages. Additionally, they have been widely adopted as language-model-as-a-service commercial APIs like GPT-4 API. However, their performance on African languages is largely unknown. We present an analysis of three popular large language models (mT0, LLaMa 2, and GPT-4) on five tasks (news topic classification, sentiment classification, machine translation, question answering, and named entity recognition) across 30 African languages, spanning different language families and geographical regions. Our results suggest that all LLMs produce below-par performance on African languages, and there is a large gap in performance compared to high-resource languages like English most tasks. We find that GPT-4 has an average or impressive performance on classification tasks but very poor results on generative tasks like machine translation. Surprisingly, we find that mT0 had the best overall on cross-lingual QA, better than the state-of-the-art supervised model (i.e. fine-tuned mT5) and GPT-4 on African languages. Overall, LLaMa 2 records the worst performance due to its limited multilingual capabilities and English-centric pre-training corpus. In general, our findings present a call-to-action to ensure African languages are well represented in large language models, given their growing popularity

Privacy Guarantees for De-identifying Text Transformations

Machine Learning approaches to Natural Language Processing tasks benefit from a comprehensive collection of real-life user data. At the same time, there is a clear need for protecting the privacy of the users whose data is collected and processed. For text collections, such as, e.g., transcripts of voice interactions or patient records, replacing sensitive parts with benign alternatives can provide de-identification. However, how much privacy is actually guaranteed by such text transformations, and are the resulting texts still useful for machine learning? In this paper, we derive formal privacy guarantees for general text transformation-based de-identification methods on the basis of Differential Privacy. We also measure the effect that different ways of masking private information in dialog transcripts have on a subsequent machine learning task. To this end, we formulate different masking strategies and compare their privacy-utility trade-offs. In particular, we compare a simple redact approach with more sophisticated word-by-word replacement using deep learning models on multiple natural language understanding tasks like named entity recognition, intent detection, and dialog act classification. We find that only word-by-word replacement is robust against performance drops in various tasks.Comment: Proceedings of INTERSPEECH 202

Efeito do nitrogênio mineralizado através da decomposição de resíduos vegetais na absorção de milho (Zea mays L.) na zona ecológica da sava-na sudanesa da Nigéria

A field experiment was designed to determine the effect of mineralized nitrogen (N) through the decomposition of leafy biomass of agroforestry tree species as residues to underscore its uptake by maize under Sudan savannah conditions. The experiment was laid out as 3 x 4 x 2 factorial in a split-split plot design with three replicates for two cropping seasons. The factors considered include: control, biomass species (Albizia lebbeck and Parkia biglobosa) as main plots, four levels of nitrogen fertilizer (0, 40, 80, 120 kg N ha-1) as sub-plots, and two maize varieties (DMR-ESR-7 and 2009 EVAT) as sub-sub plots. Data were analysed using Analysis of Variance (ANOVA). Chemical composition of A. lebbeck biomass had higher average contents of N (32.4 g kg-1) and C (186.4 g kg-1) and lower average C: N ratio (57.5) than P. biglobosa and this affected their decomposition rates, hence, A. lebbeck decomposed faster than P. biglobosa. 56 % of N in the litter bags were released within the first 2 weeks of biomass incorporation and progressively increased up to 10 weeks after planting (WAP). Total N uptake by maize was lowest (2.8 kg N ha-1) in P. biglobosa and was highest (8.6 kg N ha-1) in A. lebbeck amended plots. It is then concluded that total N uptake by maize crop increased rapidly between 4-6 WAP, and the impact was obvious in plots amended with A. lebbeck biomass than in P. biglobosa plots during the two cropping seasons.Um experimento de campo foi projetado para determinar o efeito do nitrogênio mineralizado (N) através da decomposição da biomassa foliar de espécies de árvores agroflorestais como resíduos para destacar sua absorção pelo milho em condições de savana do Sudão. O experimento foi estabelecido como fatorial 3 x 4 x 2 em um delineamento de parcelas subdivididas com três repetições para duas safras. Os fatores considerados incluem: controle, espécies de biomassa (Albizia lebbeck e Parkia biglobosa) como parcelas principais, quatro níveis de fertilizante nitrogenado (0, 40, 80, 120 kg N ha-1) como subparcelas e duas variedades de milho (DMR -ESR-7 e EVAT 2009) como sub-subparcelas. Os dados foram analisados por meio de Análise de Variância (ANOVA). A composição química da biomassa de A. lebbeck apresentou maiores teores médios de N (32,4 g kg-1) e C (186,4 g kg-1) e menor relação C:N média (57,5) do que P. biglobosa e isso afetou suas taxas de decomposição, portanto, A. lebbeck se decompôs mais rápido que P. biglobosa. 56% do N nos sacos de lixo foram liberados nas primeiras 2 semanas de incorporação da biomassa e aumentaram progressivamente até 10 semanas após o plantio (WAP). A absorção total de N pelo milho foi menor (2,8 kg N ha-1) em P. biglobosa e maior (8,6 kg N ha-1) em parcelas corrigidas de A. lebbeck. Conclui-se então que a absorção total de N pela cultura do milho aumentou rapidamente entre 4-6 WAP, e o impacto foi óbvio em parcelas alteradas com biomassa de A. lebbeck do que em parcelas de P. biglobosa durante as duas safras

E\mathcal{E} K\'U [MASK]: Integrating Yor\`ub\'a cultural greetings into machine translation

This paper investigates the performance of massively multilingual neural machine translation (NMT) systems in translating Yor\`ub\'a greetings ($\mathcal{E}$ k\'u [MASK]), which are a big part of Yor\`ub\'a language and culture, into English. To evaluate these models, we present IkiniYor\`ub\'a, a Yor\`ub\'a-English translation dataset containing some Yor\`ub\'a greetings, and sample use cases. We analysed the performance of different multilingual NMT systems including Google and NLLB and show that these models struggle to accurately translate Yor\`ub\'a greetings into English. In addition, we trained a Yor\`ub\'a-English model by finetuning an existing NMT model on the training split of IkiniYor\`ub\'a and this achieved better performance when compared to the pre-trained multilingual NMT models, although they were trained on a large volume of data.Comment: C3NLP Workshop @ EACL202

Estimating community feedback effect on topic choice in social media with predictive modeling

Social media users post content on various topics. A defining feature of social media is that other users can provide feedback—called community feedback—to their content in the form of comments, replies, and retweets. We hypothesize that the amount of received feedback influences the choice of topics on which a social media user posts. However, it is challenging to test this hypothesis as user heterogeneity and external confounders complicate measuring the feedback effect. Here, we investigate this hypothesis with a predictive approach based on an interpretable model of an author’s decision to continue the topic of their previous post. We explore the confounding factors, including author’s topic preferences and unobserved external factors such as news and social events, by optimizing the predictive accuracy. This approach enables us to identify which users are susceptible to community feedback. Overall, we find that 33% and 14% of active users in Reddit and Twitter, respectively, are influenced by community feedback. The model suggests that this feedback alters the probability of topic continuation up to 14%, depending on the user and the amount of feedback

Demographic Inference and Representative Population Estimates from Multilingual Social Media Data

Social media provide access to behavioural data at an unprecedented scale and granularity. However, using these data to understand phenomena in a broader population is difficult due to their non-representativeness and the bias of statistical inference tools towards dominant languages and groups. While demographic attribute inference could be used to mitigate such bias, current techniques are almost entirely monolingual and fail to work in a global environment. We address these challenges by combining multilingual demographic inference with post-stratification to create a more representative population sample. To learn demographic attributes, we create a new multimodal deep neural architecture for joint classification of age, gender, and organization-status of social media users that operates in 32 languages. This method substantially outperforms current state of the art while also reducing algorithmic bias. To correct for sampling biases, we propose fully interpretable multilevel regression methods that estimate inclusion probabilities from inferred joint population counts and ground-truth population counts. In a large experiment over multilingual heterogeneous European regions, we show that our demographic inference and bias correction together allow for more accurate estimates of populations and make a significant step towards representative social sensing in downstream applications with multilingual social media